With respect to remotely controlled devices, there is a need to relay back information to operators. For example, applications may include robotic tele-operation for EOD Unmanned Ground Vehicles (UGVs), general military robotic platforms or manipulators, medical manipulators, personal gaming systems, medical prosthetics and the like. Conventionally, feedback is typically video and audio, and sometimes, physical feedback may be included, so-called force feedback or haptic technology. Haptic technology, or haptics, is a tactile feedback technology which takes advantage of the sense of touch by applying forces, vibrations, or motions to the user. This mechanical stimulation may be used to assist in the creation of virtual objects in a computer simulation, to control such virtual objects, and to enhance the remote control of machines and devices (tele-robotics). Haptic devices may incorporate tactile sensors that measure forces exerted by the user on the interface. Conventional haptic devices may use vibrations directed to an operator's hands or the like. There is a need for a more intuitive apparatus and method for conveying force feedback.
Conventionally, there are a number of ways to collect force feedback information at the remotely controlled end and relay them back to the operator. Some of the other known ways to provide this force feedback to the operator are visual, vibrations at an input controller, joystick controllers applying a corresponding force, and puppet controllers applying a corresponding force.
First, force feedback may include displaying some type of graphical depiction of the force applied at the remotely controlled end. For example, a colored slider bar that moves from one end to the other correlated to the amount of force applied on the other end. This method requires the operator to divert his attention from the task he is performing and periodically “check” the visual indicator to see if what he is inputting at the controller is being applied remotely. Splitting attention between display video and a visual indicator could cause the operator to miss something or extend the length of time a particular remote task takes. If there are multiple force feedback sensors relaying information back to the operator simultaneously, the cognitive burden to watch multiple graphics on top of a video display increases substantially.
A vibrating hand controller is another method used to relay force feedback information to the operator. The intensity of these vibrations correlate to the amount of force applied at the remotely controlled end. This controller is typically used to tele-operate various functions of the remote system. One of the major drawbacks of this approach is desensitization. Since so many tasks are demanded of the operator using his hands, it may become difficult to pick up on various levels of intensity of vibrations. In addition, this force feedback method is completely tied to the input controller, where using a different control method means losing the ability to receive force feedback (or develop a different way to pass it through). Another drawback is that this method does not lend itself very well to tele-operating multiple systems that support force feedback, such as dual arm manipulators (or grippers on the manipulators) aboard a robotic platform. Using one hand controller would be difficult to output force feedback and intensity of that force from both manipulators.
Haptic feedback joysticks exist that exert a retarding force opposite the direction that the operator is commanding. With this approach, typically the more force that is exerted at the remote end (for example with a manipulator lifting a weight) is translated back to the user, pushing back in the opposite direction or not allowing joystick movement past a certain point. One drawback to this approach is it not scalable, and may be interpreted differently from user to user. It also may be difficult to pick up on force feedback because of the burden already induced on the operator as it is a control mechanism first. In other words it may suffer from desensitization while operating, making it less intuitive to use. As with other approaches, it is explicitly tied to a specific controller. Using a different input device means losing the ability to receive force feedback.
Puppet controllers typically refer to devices that mimic a remote device in degrees of freedom (of movement) and sometimes scaled link length. With this approach, the operator may move this puppet controller to a certain position and the remote platform will mimic that same orientation. With puppet controllers, you may also incorporate a scaled force resistance that corresponds to the resisting force the remote system sees. For example, with a puppet controller for a manipulator arm, “pinching” its puppet gripper fingers cause the remote manipulator to open or close. If the grippers grasp an object and it sends force feedback information back to the puppet controller, the operator feels resistance in pinching that gripper. One drawback to using this approach is that it works best with a one to one relationship, where the gripper force felt at the remote end is identical to that applied at the puppet controller gripper. If it is not 1:1, the force feedback becomes less intuitive to the operator. Similar to the vibrating controller, the force feedback is exclusively tied to this puppet controller. Using a different input device means losing the ability to receive force feedback. Another drawback is the added complexity of using multiple force feedback systems simultaneously, such as with dual arm manipulators aboard a robotic platform. The cognitive load for the operator may be significantly higher controlling so many inputs (and receiving force feedback information) at once.